Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing

摘要

An emerging class of data-intensive applications involve the geographicallydispersed extraction of complex scientific information from very largecollections of measured or computed data. Such applications arise, for example,in experimental physics, where the data in question is generated byaccelerators, and in simulation science, where the data is generated bysupercomputers. So-called Data Grids provide essential infrastructure for suchapplications, much as the Internet provides essential services for applicationssuch as e-mail and the Web. We describe here two services that we believe arefundamental to any Data Grid: reliable, high-speed transporet and replicamanagement. Our high-speed transport service, GridFTP, extends the popular FTPprotocol with new features required for Data Grid applciations, such asstriping and partial file access. Our replica management service integrates areplica catalog with GridFTP transfers to provide for the creation,registration, location, and management of dataset replicas. We present thedesign of both services and also preliminary performance results. Ourimplementations exploit security and other services provided by the GlobusToolkit.